Guest Lecture: MCMC — Gibbs Sampling & Metropolis–Hastings

Code, Packages & Diagnostics

Justin Kao

🧰 Course Tools

Markdown & Quarto: Your Storytelling Superpowers

  • 🧠 R Markdown or Quarto: Make sure it knits, renders, and saves without RStudio crying.
    (If it knits, you’re officially a data scientist.)

  • 📝 Markdown Basics — Learn how to make text bold, italic, draw diagrams, or show off anything you want to demonstrate.

  • 🚀 Hello, Quarto by Dr. Mine Çetinkaya-Rundel (Duke University) — From “What’s a chunk?” to “Look, I just published my work on GitHub!”

Markov Chain Monte Carlo(MCMC)

MCMC - Gibbs sampler: the problem

So far, we have covered conjugate and semi-conjugate posteriors — these gave us nice closed forms. But what if the joint posterior looks something like this?

\[p(\boldsymbol{\theta} \mid \mathbf{x}) \propto \left(\prod_{i=1}^n \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x_i - \mu)^2}{2\sigma^2}}\right) \cdot \frac{1}{\sigma^{2(\alpha+1)}} e^{-\beta/\sigma^2} \cdot \frac{1}{\sqrt{2\pi}\tau} e^{-\frac{(\mu-m)^2}{2\tau^2}} \cdot \frac{\lambda^{a-1} e^{-b\lambda}}{1 + e^{-\gamma(\mu - \lambda\bar{x})}}\]

Direct sampling is hard — the joint posterior can live in high dimensions with no tractable form.

Key insight: even though the joint \(p(\boldsymbol{\theta} \mid \mathbf{x})\) is hard to sample from directly, if we can derive each parameter’s full conditional distribution — conditioning on all other parameters — those are often much simpler:

\[p(\theta_j \mid \theta_1, \ldots, \theta_{j-1}, \theta_{j+1}, \ldots, \theta_p, \mathbf{x}) \quad \text{for each } j = 1, \ldots, p\]

These full conditionals are just slices of the joint — and slices of a nasty joint are often recognizable distributions we know how to sample from.

So instead of tackling the full joint at once, we sample one parameter at a time — treating all others as fixed constants. Each step is a simple univariate draw.

MCMC - Gibbs sampler: the idea

Start with some initial guess \((\theta_1^{(1)}, \theta_2^{(1)})\). Then for \(t = 2, 3, \ldots\):

Step 1. Sample \(\theta_1^{(t)}\) from \(p(\theta_1 \mid \theta_2^{(t-1)}, \mathbf{x})\) — treat \(\theta_2\) as fixed, sample \(\theta_1\)

Step 2. Sample \(\theta_2^{(t)}\) from \(p(\theta_2 \mid \theta_1^{(t)}, \mathbf{x})\) — treat the new \(\theta_1\) as fixed, sample \(\theta_2\)

Each draw is a 1D problem — much easier than the full 2D joint.

Gibbs sampler: a familiar example

Let’s refer to the bivariate normal example from Dr. Li’s Lecture 6 slides.

To build intuition, consider a target we actually know:

\[\begin{pmatrix}\theta_1\\\theta_2\end{pmatrix} \sim N_2\left(\begin{pmatrix}0\\0\end{pmatrix},\ \begin{pmatrix}1 & \rho\\ \rho & 1\end{pmatrix}\right)\]

Even though we could sample this directly, suppose we only use the full conditionals:

\[\theta_1 \mid \theta_2 \sim N(\rho\theta_2,\ 1-\rho^2), \qquad \theta_2 \mid \theta_1 \sim N(\rho\theta_1,\ 1-\rho^2)\]

Why this example? This is not a posterior — it is a plain bivariate normal whose exact shape we already know. Running Gibbs here lets us verify the algorithm works, before applying it to our actual posterior where no exact answer exists.

MCMC - Gibbs sampler: why it works

The sequence \[(\theta_1^{(1)}, \theta_2^{(1)}),\ (\theta_1^{(2)}, \theta_2^{(2)}),\ \ldots\] forms a Markov chain — each draw depends only on the previous one.

Under mild conditions, this chain converges to the joint posterior as its stationary distribution:

\[(\theta_1^{(t)}, \theta_2^{(t)}) \xrightarrow{t \to \infty} p(\theta_1, \theta_2 \mid \mathbf{x})\]

This means after enough iterations, our draws are (approximately) samples from the posterior we actually want — even though we never sampled from it directly.

Burn-in: the early part of the chain (before convergence) is discarded.

Gibbs Sampler — Interactive Animation

Step: 0 / 100

MCMC - Gibbs sampler: HW5 Problem 3

Consider the model

\[ X_i \mid \mu,\lambda \overset{\mathrm{iid}}{\sim} N\left(\mu,\frac{1}{\lambda}\right), \qquad \mu\sim N(m_0,v_0), \qquad \lambda\sim \operatorname{Gamma}(a_0,b_0), \]

where \(\mu\perp\lambda\) a priori.

Let’s derive the full conditionals.

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

The full conditional distributions are:

  • \[\mu\mid\lambda,\mathbf{x} \sim N(m_n,v_n), \quad \text{where } m_n=\frac{n\lambda\bar{x}+m_0/v_0}{n\lambda+1/v_0}, \quad v_n=\left(n\lambda+\frac{1}{v_0}\right)^{-1}\]

  • \[\lambda\mid\mu,\mathbf{x} \sim \operatorname{Gamma}\left(a_0+\frac{n}{2},\; b_0+\frac{1}{2}\sum_{i=1}^n(x_i-\mu)^2\right)\]

The files school1.dat through school8.dat give weekly hours spent on homework for students sampled from eight different schools.

Let’s set up our parameters first: \(m_0=5, v_0=0.5, a_0=1, b_0=4\)

library(coda)

# Parameters
m0 <- 5
v0 <- 0.5
a0 <- 1
b0 <- 4

# Data
x <- scan(url("https://raw.githubusercontent.com/jkstatai/Rice_Fa26_STAT_425/main/docs/Data/school1.dat"))
n <- length(x)
xbar <- mean(x)

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

# Storage
n_iter <- 10000
mu <- numeric(n_iter)
lambda <- numeric(n_iter)

# Initial values
mu[1] <- xbar
lambda[1] <- 1 / var(x)

# Gibbs sampling
for (t in 2:n_iter) {
  # 1. Sample mu | lambda, x
  vn <- 1 / (n * lambda[t-1] + 1/v0)
  mn <- vn * (n * lambda[t-1] * xbar + m0/v0)
  mu[t] <- rnorm(1, mn, sqrt(vn))
  
  # 2. Sample lambda | mu, x
  an <- a0 + n/2
  bn <- b0 + 0.5 * sum((x - mu[t])^2)
  lambda[t] <- rgamma(1, shape = an, rate = bn)
}

# Burn-in and analysis
burn_in <- 2000
mu_samp <- mu[(burn_in + 1):n_iter]
lambda_samp <- lambda[(burn_in + 1):n_iter]

mu_mcmc <- mcmc(mu_samp)
lambda_mcmc <- mcmc(lambda_samp)

MCMC - Diagnostics: did the chain converge?

After running Gibbs, we need to check the chain actually worked. Three key tools:

Trace plot — plot \(\mu^{(t)}\) and \(\lambda^{(t)}\) over iterations. A good chain looks like a “hairy caterpillar” — mixing freely, not drifting or getting stuck.

ACF (Autocorrelation Function) — measures how correlated consecutive draws are. Good mixing = ACF drops to zero quickly after lag 0.

Effective Sample Size (ESS) — because MCMC draws are dependent, 8000 draws \(\neq\) 8000 independent samples. ESS tells you the equivalent number of independent draws.

\[\text{ESS} = \frac{M}{1 + 2\sum_{k=1}^\infty \text{ACF}(k)}\]

For \(\mu\): ESS ≈ 4885. For \(\lambda\): ESS ≈ 4811. Both are high — the chain mixed well.

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

effectiveSize(mcmc(mu_samp))
    var1 
4778.017 
effectiveSize(mcmc(lambda_samp))
    var1 
4900.719 
par(mfrow = c(1, 2))
acf(mu_samp, main = "ACF of mu")
acf(lambda_samp, main = "ACF of lambda")

par(mfrow = c(1, 1))

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

plot(mu_mcmc, main = "Trace & Density of mu")

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

plot(lambda_mcmc, main = "Trace & Density of lambda")

MCMC - Gibbs sampler: HW5 Problem 3, cont’d

Evaluate the marginal posterior density \(p(\lambda \mid \mathbf{x})\) analytically up to a normalizing constant and overlay this density on your plot.

MCMC - Gibbs sampler & MH(package(coda))

  • You can technically wrap any numeric vector or matrix(object here) in mcmc_obj <- mcmc(object) , even if it isn’t a “real” MCMC sequence. However - and this is key - doing so doesn’t make it an MCMC chain in a statistical sense. It just labels it as one so that functions like plot(), acf(), and effectiveSize() can run without error.
Function Purpose
effectiveSize(mcmc_obj) Computes Effective Sample Size (ESS) — measures chain independence.
gelman.diag(mcmc.list_obj) Gelman–Rubin diagnostic for checking convergence across multiple chains.
geweke.diag(mcmc_obj) Geweke diagnostic for within-chain convergence (compares early vs. late samples).
heidel.diag(mcmc_obj) Heidelberger–Welch test for stationarity and convergence.
raftery.diag(mcmc_obj) Raftery–Lewis diagnostic estimating chain length needed for precision.

MCMC - Metropolis-Hastings Algorithm

Let’s recap of Key Concepts

  • Step 1: Propose a candidate sample from the proposal (transition) distribution
    \[ g(x_{t+1} \mid x_t) \] For example, if the proposal is Normal,
    \[ g(x_{t+1} \mid x_t) = \mathcal{N}(x_t, \sigma^2) \]

  • Step 2: Accept the proposed value with probability
    \[ A(x_t \rightarrow x_{t+1}) \]

Detail Balance Condition to reach stationary: \(\quad p(a) T(a \rightarrow b)=p(b) T(b \rightarrow a), \quad \forall a, b\)

\[ \begin{aligned} \frac{f(a)}{NC} g(b \mid a) A(a \rightarrow b)=\frac{f(b)}{NC } g(a \mid b) A(b \rightarrow a) \end{aligned} \]

MCMC - Metropolis-Hastings Algorithm

\[ \begin{aligned} \frac{f(a)}{NC} g(b \mid a) A(a \rightarrow b)=\frac{f(b)}{NC } g(a \mid b) A(b \rightarrow a) \end{aligned} \] Rewrite the balance condition:

\[\frac{A(a \rightarrow b)}{A(b \rightarrow a)}=\underbrace{\frac{f(b)}{f(a)}}_{r_f} \times \underbrace{\frac{g(a \mid b)}{g(b \mid a)}}_{r_g}\]

  • Scenario 1: \(r_f r_g<1, A(a \rightarrow b)=r_f r_g, A(b \rightarrow a)=1\)

  • Scenario 1: \(r_f r_g>1, A(a \rightarrow b)= 1, A(b \rightarrow a)=\frac{1}{r_f r_g}\)

So \(A(a \rightarrow b)=\operatorname{min}\left(r_f r_g, 1\right)\)

MCMC - Metropolis-Hastings Algorithm - Intuition

Let’s assume the distribution is symmetric

\[ \frac{A(a \rightarrow b)}{A(b \rightarrow a)}=\underbrace{\frac{f(b)}{f(a)}}_{r_f} \times \underbrace{\frac{g(a \mid b)}{g(b \mid a)}}_{r_g} = \underbrace{\frac{f(b)}{f(a)}}_{r_f} = \frac{p(b)}{p(a)} = \frac{p\left(\theta^* \mid y\right)}{p\left(\theta^{(t-1)} \mid y\right)} = r \]

\(A(a \rightarrow b)=\operatorname{min}\left(1, \frac{f(b)}{f(a)}\right) =\operatorname{min}\left(1, \frac{p(b)}{p(a)}\right) = \operatorname{min}\left(1, \frac{p\left(\theta^* \mid y\right)}{p\left(\theta^{(t-1)} \mid y\right)}=r\right)\)

  • If \(r>1\):

    • Intuition: Since \(\theta^{(t-1)}\) is already in our set, we should include \(\theta^*\) as it has a higher probability than \(\theta^{(t-1)}\).
    • Procedure: Accept \(\theta^*\) into our set, i.e. set \(\theta^{(s)}=\theta^*\).
  • If \(r<1\) :

    • Intuition: The relative frequency of \(\theta\)-values in our set equal to \(\theta^*\) compared to those equal to \(\theta^{(t-1)}\) should be \(p\left(\theta^* \mid y\right) / p\left(\theta^{(t-1)} \mid y\right)=r\). This means that for every instance of \(\theta^{(t-1)}\), we should have only a “fraction” of an instance of a \(\theta^*\) value.
    • Procedure: Set \(\theta^{(t)}\) equal to either \(\theta^*\) or \(\theta^{(t-1)}\), with probability \(r\) and \(1-r\) respectively.

MCMC - Metropolis-Hastings Algorithm

Consider the file school1. dat which contains data on the amount of times students at a high school spent on studying or homework during an exam period. Suppose we fit the following model:

\[ \begin{aligned} Y_i & \sim \mathcal{N}\left(\mu, \sigma^2\right), \quad i=1, \ldots, n \\ \mu & \sim \mathcal{N}\left(\mu_0, \tau_0^2\right) \end{aligned} \]

with \(\mu_0=5, \tau_0^2=0.5\). Let us fix \(\sigma^2=1\). Implement a Metropolis-Hastings:

library(coda)

# Read data
y <- scan(url("https://raw.githubusercontent.com/jkstatai/Rice_Fa26_STAT_425/main/docs/Data/school1.dat"))
n <- length(y)

# Known hyperparameters
mu0 <- 5
tau0_sq <- 0.5
sigma_sq <- 1

# Log-posterior: log p(mu | y) ∝ log p(y | mu) + log p(mu)
log_post <- function(mu, y, mu0, tau0_sq, sigma_sq) {
  log_lik <- sum(dnorm(y, mean = mu, sd = sqrt(sigma_sq), log = TRUE))
  log_prior <- dnorm(mu, mean = mu0, sd = sqrt(tau0_sq), log = TRUE)
  log_lik + log_prior
}

MCMC - Metropolis-Hastings Algorithm, cont’d

  • A proposal function for \(\mu\) centered at the previously sampled value.

  • My proposal \(\mu^* \sim q\left(\mu^* \mid \mu^{(0)}\right)=N\left(\mu^{(0)}, \sigma_p^2\right)\), where we can set \(\mu^{(0)} = \bar{y}\)

mean(y)
[1] 9.464
  • \(\mu^*=\mu^{(t-1)}+\varepsilon, \quad \varepsilon \sim N\left(0, \sigma_p^2\right)\) Hence, \(q\left(\mu^* \mid \mu^{(t-1)}\right)=N\left(\mu^{(t-1)}, \sigma_p^2\right)\)
# Metropolis–Hastings sampler (Random-Walk proposal)
mh_sampler <- function(sigma_p, n_iter = 5000, burn_in = 1000) {
  mu <- numeric(n_iter)
  mu[1] <- mean(y)
  
  for (t in 2:n_iter) {
    mu_prop <- rnorm(1, mean = mu[t - 1], sd = sigma_p)
    log_r <- log_post(mu_prop, y, mu0, tau0_sq, sigma_sq) -
             log_post(mu[t - 1], y, mu0, tau0_sq, sigma_sq)
    if (log(runif(1)) < log_r) {
      mu[t] <- mu_prop
    } else {
      mu[t] <- mu[t - 1]
    }
  }
  
  # Drop burn-in and return MCMC object
  chain <- mcmc(mu[(burn_in + 1):n_iter])
  return(chain)
}

MCMC - Metropolis-Hastings Algorithm, cont’d

# Run chain with sigma_p = 0.5
set.seed(425)
chain_small <- mh_sampler(sigma_p = 0.5)

# Summaries and plots
summary(chain_small)

Iterations = 1:4000
Thinning interval = 1 
Number of chains = 1 
Sample size per chain = 4000 

1. Empirical mean and standard deviation for each variable,
   plus standard error of the mean:

          Mean             SD       Naive SE Time-series SE 
      9.132470       0.193392       0.003058       0.006434 

2. Quantiles for each variable:

 2.5%   25%   50%   75% 97.5% 
8.768 9.002 9.133 9.264 9.505 

MCMC - Metropolis-Hastings Algorithm, contd)

traceplot(chain_small, main = "Trace Plot (sigma_p = 0.5)")

MCMC - Metropolis-Hastings Algorithm, contd)

densplot(chain_small, main = "Posterior Density (sigma_p = 0.5)")

MCMC - Metropolis-Hastings Algorithm, cont’d

autocorr.plot(chain_small, main = "Autocorrelation (sigma_p = 0.5)")

MCMC - Metropolis-Hastings Algorithm, cont’d

(b) Provide a trace plot and a plot of the histogram and/or kernel density estimate for each chain.

To answer this, you can run the following code and then compare the plots:

set.seed(123)
chain1 <- mh_sampler(sigma_p = 0.5)
set.seed(456)
chain2 <- mh_sampler(sigma_p = 0.5)
set.seed(789)
chain3 <- mh_sampler(sigma_p = 0.5)
  • summary(),autocorr.plot(), densplot()
  • I have demonstrated the case when \(\sigma_p=0.5\). Don’t forget to try \(\sigma_p=2\) !

MCMC - Metropolis-Hastings Algorithm, cont’d

(c) After removing an appropriate burn-in, evaluate the acceptance rate and the autocorrelation for varying lags. Which value of \(\sigma_p\) provides a better choice for the proposal function?

# Acceptance rate function
acceptance_rate <- function(chain) {
  draws <- as.numeric(chain)
  mean(diff(draws) != 0)
}
acceptance_rate(chain_small)
[1] 0.4211053

(This is just an arbitrary number I came up with — you should check your own chain.)

The pattern is likely to be:

  • \(\sigma_p=0.5\) accepts \(\sim 82 \%\) proposals \(\rightarrow\) too conservative.
  • \(\sigma_p=2\) accepts \(\sim 27 \%\) proposals \(\rightarrow\) better exploration.

MCMC - Gibbs sampler & MH(package(coda))

  • You can technically wrap any numeric vector or matrix(object here) in mcmc_obj <- mcmc(object) , even if it isn’t a “real” MCMC sequence. However - and this is key - doing so doesn’t make it an MCMC chain in a statistical sense. It just labels it as one so that functions like plot(), acf(), and effectiveSize() can run without error.
Function Purpose
effectiveSize(mcmc_obj) Computes Effective Sample Size (ESS) — measures chain independence.
gelman.diag(mcmc.list_obj) Gelman–Rubin diagnostic for checking convergence across multiple chains.
geweke.diag(mcmc_obj) Geweke diagnostic for within-chain convergence (compares early vs. late samples).
heidel.diag(mcmc_obj) Heidelberger–Welch test for stationarity and convergence.
raftery.diag(mcmc_obj) Raftery–Lewis diagnostic estimating chain length needed for precision.

Finally

Resources

Thank you!

Have a great rest of your day!